{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AI email assistant with Composio and LanceDB"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This guide demonstrates how to integrate Composio’s Gmail tool with LanceDB to create an agent that will automatically respond to emails by searching a LanceDB vectorstore for relevant information and using that information to draft replies. This setup is useful for automating email responses by leveraging a vector store that contains relevant data, such as FAQs or documentation. Composio allows an AI agent/your app to easily connect to your user's apps like Gmail, Slack, Trello etc. Paired together with LanceDB to store, manage and query data, more personalized and context-aware agents can be created."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup and Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install dependencies\n",
    "!pip install composio-langchain lancedb langchain-community langchain langchain-core langchain-openai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Composio setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start using Composio's Gmail tool, you need to create an integration between your Gmail account and Composio.\n",
    "\n",
    "This can be done simply with a command - "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!composio add gmail"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Follow the link generated by the above command to complete authentication and finish setting up the integration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To respond to emails automatically, we need to set up a trigger that listens for new emails. This can be done using the following command - "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!composio triggers enable gmail_new_gmail_message"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This enables the `gmail_new_gmail_message` trigger. Now, when a new email arrives, the trigger provides data about the email like the sender, mail content etc., which can then be used to draft an appropriate reply to the email."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use OpenAI's embedding model and LLM in this example, so you need to set your OpenAI API key as an environment variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"OPENAI_API_KEY\"] = \"your-openai-api-key\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the data\n",
    "\n",
    "Now, we'll get the data we want to store and query with our agent. For this example, we're going to use a research paper on AI, but this can be replaced with whatever docs you want to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://arxiv.org/pdf/1706.03762 -O ai.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will download the research paper from the above url and store it in a file named 'ai.pdf'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Generate embeddings and store data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we generate embeddings for the data and then store the embeddings in a LanceDB vector store -"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "from langchain_openai import OpenAIEmbeddings\n",
    "\n",
    "# Create recusrive text splitter\n",
    "text_splitter = RecursiveCharacterTextSplitter(chunk_size=700, chunk_overlap=50)\n",
    "\n",
    "embeddings = OpenAIEmbeddings()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders import PyPDFLoader\n",
    "from langchain_community.vectorstores import LanceDB\n",
    "\n",
    "# Load and split documents\n",
    "loader = PyPDFLoader(\"ai.pdf\", extract_images=False)\n",
    "docs = loader.load_and_split(text_splitter)\n",
    "\n",
    "# Add documents to LanceDB vector store\n",
    "vectorstore = LanceDB.from_documents(documents=docs, embedding=embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialize Composio's toolset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we initialize Composio's toolset and get the `GMAIL_REPLY_TO_THREAD` action. This action is basically a function that the LLM can call. Composio provides these pre-built actions with optimized JSON schemas which makes it very easy to integrate external tools with LLMs/agentic workflows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from composio_langchain import Action, ComposioToolSet\n",
    "\n",
    "# Initialize Composio's toolset\n",
    "toolset = ComposioToolSet()\n",
    "\n",
    "replyTool = toolset.get_tools(\n",
    "    actions=[\n",
    "        Action.GMAIL_REPLY_TO_THREAD,  # Reply to a gmail thread\n",
    "    ],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also define a custom tool that\tqueries the vector store based on the incoming query from a user - "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def search_db(query: str) -> str:\n",
    "    \"\"\"Searches the LanceDB vector store for user query and returns the results.\"\"\"\n",
    "\n",
    "    response = vectorstore.similarity_search(query)\n",
    "    return response"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we create a `StructuredTool` from the above function - "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.tools import StructuredTool\n",
    "\n",
    "searchTool = StructuredTool.from_function(\n",
    "    func=search_db,\n",
    "    name=\"search_db\",\n",
    "    description=\"Searches Lance DB vector store for the given query and returns the results.\",\n",
    ")\n",
    "\n",
    "# Combine the reply tool and the search tool to pass to agent\n",
    "tools: list[StructuredTool] = replyTool + [searchTool]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting up the agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import ChatOpenAI\n",
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain.agents import AgentExecutor, create_tool_calling_agent\n",
    "\n",
    "# initialize LLM\n",
    "llm = ChatOpenAI(model=\"gpt-4o-mini\")\n",
    "\n",
    "# The prompt can be customized to fit your use case\n",
    "prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\n",
    "            \"system\",\n",
    "            \"You are an AI email assistant that can write and reply to emails. You have to use the search_db tool to search the lance vector store for the user's query. When the user asks you a question, use the search_db tool to search the lance vector store and then answer the question using the search results. Send the answer back to the user in an email.\",\n",
    "        ),\n",
    "        (\"human\", \"{input}\"),\n",
    "        (\"placeholder\", \"{agent_scratchpad}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "agent = create_tool_calling_agent(\n",
    "    llm=llm,\n",
    "    tools=tools,\n",
    "    prompt=prompt,\n",
    ")\n",
    "\n",
    "# Create an instance of AgentExecutor\n",
    "agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we create a listener for the trigger that we initialized in the beginning - "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the listener\n",
    "listener = toolset.create_trigger_listener()\n",
    "\n",
    "\n",
    "# Attach listener to trigger and provide callback function that is executed when the trigger receives new data\n",
    "@listener.callback(filters={\"trigger_name\": \"GMAIL_NEW_GMAIL_MESSAGE\"})\n",
    "def on_new_gmail_message(event) -> None:\n",
    "    try:\n",
    "        print(\"data received - \", event)\n",
    "\n",
    "        # Extract the relevant information from the event\n",
    "        payload = event.payload\n",
    "        thread_id = payload.get(\"threadId\")\n",
    "        message = payload.get(\"messageText\")\n",
    "        sender_mail = payload.get(\"sender\")\n",
    "\n",
    "        res = agent_executor.invoke(\n",
    "            {\n",
    "                \"input\": f\"This is the query you have to respond to: {message}. It's from {sender_mail} and the threadId is {thread_id}.\"\n",
    "            }\n",
    "        )\n",
    "        print(res)\n",
    "    except Exception as e:\n",
    "        print(\"Error:\", e)\n",
    "\n",
    "\n",
    "print(\"Listener started!\")\n",
    "listener.listen()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "Listener started!\n",
    "[2024-09-18 20:05:59,715][INFO] Received trigger event with trigger ID: ea36d63f-5cc9-4581-9a19-b647e7468697 and trigger name: GMAIL_NEW_GMAIL_MESSAGE\n",
    "data received -  appName='gmail' payload={'threadId': '192058f5b7fe8...', 'messageId': '192058f5b7fe8...', 'messageTimestamp': '2024-09-18T14:35:20Z', 'messageText': 'What are encoders?\\r\\n'...}\n",
    "\n",
    "> Entering new AgentExecutor chain...\n",
    "\n",
    "Invoking: `search_db` with `{'query': 'What are encoders?'}`\n",
    "\n",
    "[Document(metadata={'page': 0, 'source': 'ai.pdf'}, page_content='convolutional neural networks that include an encoder and a decoder. The best\\nperforming models also connect the encoder and decoder through an attention\\nmechanism. We propose a new simple network architecture, the Transformer,\\nbased solely on attention mechanisms, dispensing with recurrence and convolutions\\nentirely. Experiments on two machine translation tasks show these models to\\nbe superior in quality while being more parallelizable and requiring significantly\\nless time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-\\nto-German translation task, improving over the existing best results, including'), Document(metadata={'page': 1, 'source': 'ai.pdf'}, page_content='3 Model Architecture\\nMost competitive neural sequence transduction models have an encoder-decoder structure [ 5,2,35].\\nHere, the encoder maps an input sequence of symbol representations (x1, ..., x n)to a sequence\\nof continuous representations z= (z1, ..., z n). Given z, the decoder then generates an output\\nsequence (y1, ..., y m)of symbols one element at a time. At each step the model is auto-regressive\\n[10], consuming the previously generated symbols as additional input when generating the next.\\n2'), Document(metadata={'page': 4, 'source': 'ai.pdf'}, page_content='[38, 2, 9].\\n•The encoder contains self-attention layers. In a self-attention layer all of the keys, values\\nand queries come from the same place, in this case, the output of the previous layer in the\\nencoder. Each position in the encoder can attend to all positions in the previous layer of the\\nencoder.\\n•Similarly, self-attention layers in the decoder allow each position in the decoder to attend to\\nall positions in the decoder up to and including that position. We need to prevent leftward\\ninformation flow in the decoder to preserve the auto-regressive property. We implement this\\ninside of scaled dot-product attention by masking out (setting to −∞) all values in the input'), Document(metadata={'page': 1, 'source': 'ai.pdf'}, page_content='1 Introduction\\nRecurrent neural networks, long short-term memory [ 13] and gated recurrent [ 7] neural networks\\nin particular, have been firmly established as state of the art approaches in sequence modeling and\\ntransduction problems such as language modeling and machine translation [ 35,2,5]. Numerous\\nefforts have since continued to push the boundaries of recurrent language models and encoder-decoder\\narchitectures [38, 24, 15].\\nRecurrent models typically factor computation along the symbol positions of the input and output\\nsequences. Aligning the positions to steps in computation time, they generate a sequence of hidden')]\n",
    "\n",
    "Invoking: `GMAIL_REPLY_TO_THREAD` with `{'thread_id': '192058f5b7fe8...', 'message_body': 'Dear John,\\n\\nEncoders are a crucial component of many neural network architectures, particularly in sequence modeling tasks. In an encoder-decoder structure, the encoder maps an input sequence of symbols into a sequence of continuous representations. This allows the model to capture the underlying features of the input data.\\n\\nFor example, in the case of machine translation, the encoder processes the input sentence and generates a representation that the decoder then uses to produce the translated output sentence.\\n\\nThe encoder often includes self-attention layers, which allow it to consider all positions in the input when creating its representations. This mechanism is part of what makes modern models like Transformers so effective.\\n\\nIf you have any more questions or need further clarification, feel free to ask!\\n\\nBest regards,\\n[Your Name]', 'recipient_email': 'example@gmail.com'}`\n",
    "\n",
    "\n",
    "[2024-09-18 20:06:11,352][INFO] Executing `GMAIL_REPLY_TO_THREAD` with params={'thread_id': '192058f5b7fe8d17', 'message_body': 'Dear John,\\n\\nEncoders are a crucial component of many neural network architectures, particularly in sequence modeling tasks. In an encoder-decoder structur...\n",
    "[2024-09-18 20:06:13,109][INFO] Got response={'successfull': True, 'data': {'response_data': {'id': '192058ff6bbec...', 'threadId': '192058f5b7fe8...', 'labelIds': ['SENT']}}, 'error': None} from action=<composio.client.enums._action.Action object at 0x7f753839ea70> with params={'thread_...\n",
    "{'successfull': True, 'data': {'response_data': {'id': '192058ff6bbec...', 'threadId': '192058f5b7fe8...', 'labelIds': ['SENT']}}, 'error': None}I've replied to John's email regarding encoders. If you need anything else, feel free to ask!\n",
    "\n",
    "> Finished chain."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Connect with Composio and learn more"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you encounter any problems, please let us know at our [Discord](https://discord.com/invite/cNruWaAhQk).\n",
    "\n",
    "Check out [Composio's documentation](https://docs.composio.dev/introduction/intro/overview) to learn more about how to use and integrate various tools for different usecases."
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
